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Abstract — A new property which relies on the linear program- 
ming (LP) decoder, the approximate maximum-likelihood certifi- 
cate (AMLC), is introduced. When using the belief propagation 
decoder, this property is a measure of how close the decoded 
codeword is to the LP solution. Using upper bounding techniques, 
it is demonstrated that the conditional frame error probability 
given that the AMLC holds is, with some degree of confidence, 
below a threshold. In channels with low noise, this threshold is 
several orders of magnitude lower than the simulated frame error 
rate, and our bound holds with very high degree of confidence. 
In contrast, showing this error performance by simulation would 
require very long Monte Carlo runs. When the AMLC holds, our 
approach thus provides the decoder with extra error detection 
capability, which is especially important in applications requiring 
high data integrity. 



I. Introduction 

Linear programming (LP) decoding has emerged in recent 
years as a potential candidate algorithm for approximating 
maximum-likelihood (ML) decoding. One reason for this is 
that it has been shown fV\ that the LP decoding algorithm has 
the ML certificate property, i.e., that if the decoder outputs a 
valid codeword, it is guaranteed to be the ML codeword. 

Since the discovery of LP decoding, several papers have 
been written on the subject of improving the performance 
of the decoder, e.g., by using integer programming or mixed 
integer linear programming [1], [2 |, adding constraints to the 
Tanner graph [3| and guessing facets of the poly tope |4]. 
Moreover, the issue of decoding complexity has been ad- 
dressed (5], fSl, fT], as the complexity of linear programming 
techniques is in general polynomial but not linear in the block 
length N. Vontobel and Koetter jS) have proposed an iterative, 
Gauss-Seidel-type algorithm for approximate LP decoding. 
Based on their general approach, a linear-complexity (0{N)) 
iterative approximate decoder [8] was suggested. 

This low-complexity LP decoder was recently put to use 
in a framework [9| aimed at harnessing the LP decoder for 
tasks other than decoding. In this context, an algorithm with 
complexity O(iV^) was proposed which produces a lower 
bound on the minimum distance of a specific code. Another 
use is an algorithm of the same complexity for finding a tight 
lower bound on the fractional distance. 

In this paper we propose a new application for using the 
LP decoder by introducing a new concept, the approximate 
ML certificate (AMLC), a tool which can improve the error 
detection capability of the belief propagation (BP) decoder 



We show that if the BP decoder output satisfies the AMLC 
property (in particular, it is a codeword), then there is a 
high degree of certainty that it is the correct codeword. It 
is demonstrated that, when applying this technique within the 
error floor region, the frame error rate implied by the AMLC 
is several orders of magnitude lower than the average rate (the 
average rate in the error floor region was previously studied 
by Richardson ifTOI ): ascertaining this improved reliability 
directly using Monte Carlo simulation would require very long 
simulation runs. This makes the AMLC especially useful in 
applications where a high level of data integrity is required. 

The LP decoder is a central component in the evaluation 
of the AMLC, as is the aforementioned 0{N^) algorithm for 
obtaining a lower bound on the minimum distance. Another 
component used in our analysis is the generalized second 
version of the Duman-Salehi bound, as derived by Sason and 
Shamai |1 11 ; this bound is an upper bound on the ML decoding 
probability. A slightly modified version of this bound is used 
for our purposes. 

This paper is organized as follows. Section HDprovides some 
background material, related primarily to the LP decoder 
In section |III1 we prove our main result concerning the 
approximate ML certificate. In Section|lV]numerical examples 
are provided, and the paper is concluded in Section |V] 

II. Preliminaries 

Consider an LDPC code C described by a Tanner graph 
with N variable nodes and M check nodes. A codeword 
c G C is transmitted over a discrete memoryless binary-input, 
output-symmetric (MBIOS) channel described by a probability 
transition function Q{y \ c), where c is the transmitted code- 
bit and y is the channel output (we will also use the vector 
notation Q(y | c) where y is the channel output vector and c 
is the transmitted codeword, the meaning will be clear from 
the context). Following the notation in fT], let I and J' be the 
sets of variable and constraint nodes, respectively, such that 
\I\ = iV and I J'l = M. Define the set A/j to be the set of 
neighbors of variable node i £ I. Similarly, Afj is the set of 
neighbors of check node j £ J . Denote by Cj the constituent 
binary single parity check code corresponding to node j G J . 
Let G C denote the all-zero codeword. 

The LP decoder UJ solves the following optimization prob- 
lem. 
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(A, Ao;) — argminP(c) 

(c,i^) 
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(i.e., A is the optimal c-vector and Xu: is the optimal u;-vector) 
subject to 

Wj^g > yj ej , geCj (2) 



sec, 



(3) 



c, = ^ wj^g e J , ie Nj (4) 

where the vector u is defined by 
and where 

P(c)=^c,7, (5) 



and 



A , Q{y^ I 0) 
7» = log ■ 



Q{y^ 1 1) 

is the log-likelihood ratio (LLR). All logarithms are natural. 

An important observation is that the LP decoder has the ML 
certificate |1| in the sense that if the solution A is integer (in 
fact an integer Xijj implies that A is also integer by dUi) then 
it is the ML decision. 

III. The approximate ML certificate property 

Let the binary LDPC code C be selected uniformly from an 
ensemble of LDPC codes C° (e.g., the ensemble of all (c, d)- 
regular codes) and assume C has M codewords. Let Cm G 
C,m € {1, . . . , M} be the codeword selected for transmission 
over a MBIOS channel, and suppose that the channel output 
vector is y. The received vector y is decoded by a standard 
belief propagation (BP) decoder, which outputs an estimate c 
which may or may not be a valid codeword. Let P(A) be the 
optimal value of the LP problem. By the fact that is a 
codeword and hence feasible in the LP we have 



P(A) < P(c™) 
Now suppose that for some S > 

P(c) - P(A) < 6 



(6) 



(7) 



and that the decoder output c is a valid codeword. We call 
this event the approximate ML certificate and the constant 6 
the proximity gap. Formally, the AMLC happens if and only 
if 

c e AMLC((5) (8) 



where 



AMLC((5) = {c : c e C , P(c) - P(A) < 6} (9) 



When 5 = the AMLC coincides with the standard ML 
certificate of [1], since in this case the codeword c is the ML 
solution. By (|6l)-(|7]i we conclude that 



P(c) - P(c„) < (5 



(10) 



Now, consider the transmission of a code chosen at random 
from C°. The transmitted codeword is also selected at 
random from the chosen code. If the AMLC holds, then the 



word error probability given that c„i was transmitted can be 
upper bounded as 



Pr l^c^c 

< Pr 

Pr 

< — 



c e AMLC((5) 
c„j trans. 

3c eC 

C C^^^ 

3c eC 

C Cjji 



c G AMLC((5) 
c„, trans. 



P(c) < P(c„,) + S 
P(c) < P(cm) + 6 I c,„ ti-ans 



Pr (c e AMLC ((5) | c^ trans 
3c eC 

C 7^ Cm 



I Q(y I c) 



< S \ c,n trans. 



Pr(c e AMLC(5) I Cm trans.) 



(11) 



where in the first inequality we used (fTHt . and in the equality 
we used the fact that P(c) = log [ °j and P(c„i) = 

log (^y^i ^c'^y Note that in ( fTTT i. the numerator depends on the 
channel probability transition function and the code chosen, 
while the denominator depends on the channel, the code, 
and also on the decoding algorithm. We can further upper 
bound the expression (fTTT i by upper-bounding the numerator 
and lower-bounding the denominator In the process of doing 
so, we eliminate the dependence on the transmitted message 

TO. 



A. A lower bound on the denominator 

Consider the denominator in (fTTT i. The following lemma 
asserts its independence of the transmitted codeword. 

Lemma 1: Consider the vector A which is output by the LP 
decoder Also assume that c,„ is the transmitted codeword and 
that the BP decoder is used. Then the expression 



Pr (c e AMLC((5) | c„i trans.) 



(12) 



is independent of to, and in particular it can be assumed in 
(fT2l l that the all-zero codeword is transmitted. 

Proof: See Appendix lAl ■ 
To get a lower bound on (fTSI l. one could run Monte Carlo 
simulations. Consider a series of experiments conducted to 
estimate 77 = Pr(c S AMLC((5) | trans.). In each experi- 
ment we draw a code at random from C", transmit the all- 
zero codeword over the noisy channel and decode. Suppose 
that we run L experiments and find that c 6 AMLC(5) in Li 
experiments. Let 

L 

1=1 

where if in the i'th experiment c G AMLC ((5), then Xi = 1; 
otherwise Xi = 0. If the channel is low-noise then we would 
expect to have Li = — e) with small e, and in particular we 
would expect to have e < 0.5, which (for large L) would imply 
7] > 0.5. Since 77 is a deterministic but unknown parameter, 
we cannot claim that 77 > 0.5, even if e is small; rather, this 
situation falls under the framework of non-bayesian hypothesis 
testing, so the series of experiments does allow us to say 
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something about 77 with some degree of confidence. Consider 
the hypothesis 

Ho : T]< 0.5 (13) 
For e < 0.5, the following inequality holds 

Pr(ii > (1 - e)L I Ho valid) < 2'^ (^^^ ■ {eL + 1) (14) 

since the RHS is an upper bound on the tail of a binomial 
distribution. Now suppose that in a Monte Carlo simulation 
we get Li = L{1 — e) with e < 0.5. By ( fT4b we observe that 
if e is very small, then the RHS of ( fT4] i is very low, and thus we 
can reject Ho with a high degree of confidence. Conversely, if 
in our simulation e is close to 0.5, we cannot reject Hq with 
high confidence. 
Define 



C(i,e) = l-2- 



(eL + 1) 



(15) 



Given the Monte Carlo result discussed above, one may 
conclude that 

Pr (c 7^ c,„ I c e AMLC(J) , c™ trans.) 

3c e C 



< 2Pr 



c ^ c„ 



c„i trans 
(16) 



which reflects the assertion i] > 0.5. This assertion holds with 
confidence level ^(L, e). Note that for fixed e, the likelihood 
that the bound (fT6] i does not hold decays exponentially with 
L. 

The standard approach to estimating the frame error rate 
performance is to use a Monte Carlo simulation. The result is 
a confidence interval on the actual error rate. In our method 
we also use a Monte Carlo simulation. However, in the 
following we derive an analytic bound on the RHS of (fTSI l 
which, combined with the simulation, enables us to obtain 
extremely large confidence levels for very small frame error 
rates whenever the AMLC holds. 

B. An upper bound on the numerator 

Consider now the RHS of (fTSI l (disregarding the constant 
2). Recalling that C is chosen at random from C", one may 
write 

Q{y\ 



Pr 



3c e C 

c 7^ C„i 
^Pr 



Q(y|c 
3c e C 

C ^ 



Q{y\ 



trans. 



Q{y I Cm) 

Pr (C = Ci I c„, trans.) 



c„, trans. 



(17) 



Clearly, Pr {C = Ci \ Cm trans.) = Pr (C = Ci) as the selec- 
tion of the message is independent of the selection of the 
code. In addition, we have the following result regarding the 
independence of the inner expression in the sum (fTTt on m. 
Lemma 2: The expression 



Pr 



3c G C 

C ^ Cm 



Q{y I Cm) 



c^c. 

Cm trans. 



(18) 



appearing within the sum ( fTTj i is independent of m. 

Proof: See Appendix IbI ■ 
Due to Lemma |2l we can assume without loss of generality 
that the all-zero codeword is transmitted and rewrite (fTTt as 

3c e C Q(y I c) 



Pr 



c^O 



Q{y I 0) 
3c e C 

c 7^ 



> 1 trans. 



Q{y I c) ^ ^ 



Q(y I 0) 



trans. 



Pr (C=C,) 



(19) 

Sason and Shamai fTTl have proposed a tight upper bound 
on the ML decoding error probability using the generalized 
second version of the Duman-Salehi bound, referred to as the 
DS2 bound. Using a slightly modified version of this bound, 
we can find an upper bound on 



Pr 



3c e C 

c 7^ 



Q{y I c) 



Q(y I 0) 

To this end, one may write 

3c eC 



> 1 



C = C, 
trans. 



Pr 



c 7^ 



Q(y|c) . ^ 

-e > 1 



Q{y 
( 







<2^Q(y|o) 



\p ( Q(y|c 
^ W(y|o)^ 



c = c, 

trans. 

y 



1 c#0 

\ceCi 



Y.^l{y)-^%{y)-'Q{y\0) 



E 

\cec. 



fQ{y\' 



VQ(y|o) 



y 



*°^(y) 



^l{y)--^Q{y\oy 



\ 



,^AQ(y|o)' 

cGCi 



I 

(20) 

where the expression on the second line, which holds for all 
A, p > 0, is an adaptation of the 1965 Gallager bound [T2l| to 
our purposes, and ^'^/(y) is a probability measure on y called 
a tilting measure fTT^, which is allowed in general to depend 
on the transmitted codeword. By invoking Jensen's inequality 



in (|20|, 



Pr 



we get 

3c e C 
c 7^ 

/ 



Q(y I c) 
Q(y I 0) 



e/>l 



< 



5]5]g(y|0)i;<(y)i- 



c#0 

\ceCi 



trans. 



/ Q(y|c) , 
W(y|o) 



I 



(21) 



which holds for A>0, 0<p<l. Now let us restrict our 
discussion to tilting measures which do not depend on the 
transmitted codeword and which also decompose as TV-fold 
products of the same single-letter measure, i.e.. 



N 



n(y) = n'/'(2^') 
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Also recall that the channel is memoryless and thus also 
decomposes as an iV-fold product. Using this in (ISTT i yields 



p^, 3cec . Q(y|c) 



trans. 



< e 



SpX 



N 



N-h 



Y,A,AY,i^[yf--^Q{y\Q)-^ 



h=l 



(22) 



EV'(2/)'"^O(2/|0)'^Q(y|i)' 



where Ah is the distance spectrum of the code Ci. Now we 
partition the code Ci into constant Hamming weight subcodes 
where Ci^h contains all words in Ci of weight h (note that in 
general these subcodes are nonlinear). By applying a union 
bound over the subcodes on the LHS of ( |22] | we get 



Pr 



3c eC . Q(y|c) 



CT^o ■ 0(y|o) 

N , 

< ^ Pr 3c e C,,h ■■ 



>l\C = C, , trans. 



h=l 
N 



Q{y I 0) - 



c = c, 

trans. 



h=l 



(23) 



where by 



N-h 



Y.^{y)'-^Qiy\o)'^Q{y\i) 



(24) 



Let Ah and Pi{h) denote the averages of the distance dis- 
tribution Ah and Pi (ft.), respectively, taken over the en- 
semble C*^. By Jensen's inequality (applied as [(Ah)''] < 
[^'' ,0 < p < 1) we have 



Pi{h) <e'p\Ah) 



(25) 



The overall bound is given by 



N 



Pr (c ^ c„, I c e AMLC((5) , c„ trans.) < 2 ^ Pi (ft) 

h=l 

(26) 

where Pi (ft) is given by ( l25T l. This upper bound only depends 
on the average distance spectrum, which is known for many 
code ensembles and in particular for LDPC codes. Now, we 
can optimize the bound ( |25] ) over A>0,0<p<l and 
the tilting measure 'ip{ ). This optimization is performed for 
every value of ft separately. Some additional technical details 
regarding this optimization are provided in Appendix O 



C. Application of the AMLC to expurgated LDPC ensembles 

In this subsection we consider the application of the upper 
bound on the error probability given the AMLC to expurgated 
ensembles of LDPC codes. The expurgated ensemble C is 
obtained from the original ensemble C^ by removing all codes 
with minimum distance 7 or less. The reason for dealing 
with this ensemble rather than C° is that the decoding error 
probability over C° is dominated ifTsl . lfT4l by a small set of 
"bad" codes with small minimum distance; we will show that 
if we can avoid these "bad" codes, then the occurrence of the 
AMLC implies very low error rates. 

Let A^l^ denote the average distance spectrum over C^. It 
was shown lfT4l that if 7 > is selected small enough, then 
with probability 1 — o(l), a randomly-selected code from C*' 
is also in C; this implies that for large enough N, so that 
less than half the codes are expurgated, the following bound 

holds: 

2Ah, ft > 7 
0, ft < 7 



(27) 



When using the DS2 bound we can plug A^^ instead of Ah in 
(|25] |. In practice, when applying the Monte Carlo procedure 
outlined in Section IIII-AI we draw codes at random from C° 
and thus we need to test whether these codes are also in C. To 
do this, we use the procedure described in [9, Section 5] which 
obtains a lower bound LP(Ci) on the minimum distance of 
the randomly-drawn code Ci. If LB{Ci) > 7, then Ci ^ C. 
Note, however, that the converse is not necessarily true, i.e., 
we could have Ci G C but with LB{Ci) < 7. Define the 
ensemble 

C'' = {CeC° : LB{C) > 7} (28) 

then clearly C" C C. Let AJ^ be the average distance spectrum 
over C. We will obtain an upper bound on AJ^ which is similar 
to ( |27| i using a Monte Carlo approach, similar to the argument 
made in Section ITlI-AI Suppose we run L experiments. In each 
experiment we randomly pick a code C e C° and calculate 
LB{C). Now suppose that in L2 ~ L{1 — £2) experiments we 
obtain that LB(C) > 7, and £2 < 0.5 is small. From this set 
of experiments, we conclude as we did in Section IIII-AI that 



2Ah, 
0, 



ft > 7 
ft < 7 



(29) 



with high confidence level. 

Consider the following procedure for obtaining a bound on 
the confidence level of (|25]|-(|26]| when AJ^ (upper-bounded in 
(|29] |) is used as the distance spectrum. The confidence level 
output by this algorithm is a combination of the confidence 
level associated with 7/ > 0.5 (see Section IIII-Al i and the 
statement ( |29] ). That is, the null hypothesis in this case is 

Ho : {rj< 0.5 or Pr(LP(C) > 7) < 0.5} (30) 



Algorithm 1: Given an ensemble of codes C°, a channel 
probability distribution Q{-\-) and number of trials L, do: 

1) Initialize: Set P = 0. 

2) Loop L times: 

< Pick a code C uniformly from C°. 
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. Calculate LB{C). 

. If LB{C) < 7, + 1 and skip to next loop 

iteration. 

• Transmit the all-zero codeword through the channel. 

• Decode using the BP decoder and the LP decoder 

• If the BP decoder output c is not a codeword, or if 
P(c) - P(A) > S, set E ^ E + 1 

3) Output confidence level of bound: Define e ^ E/L. 
If e < 0.5, output ^{L, e) defined in (fTSl l. Otherwise, 
output "error". 

Algorithm [1] is introduced for the purpose of jointly assess- 
ing the possibility of rejecting the hypothesis ( fT3] l. and the 
validity of ( |29] l as an upper bound on the distance spectrum 
using the same confidence level-based Monte Carlo based 
method from Section IIII-AI The algorithm counts the number 
of failed attempts E out of L experiments, where a failure 
consists of either having a code C not pass the test LB{C) > 7, 
or, having passed this test, getting a BP decoder output which 
does not satisfy the AMLC. The algorithm is correct because 
if the null hypothesis ( |30] | holds, then in any single experiment 
we would have a probability of failure at least 0.5. If the total 
number of failures E is small (i.e., less than half the total 
number of experiments) then the confidence level, following 
the derivation in Section IIII-AI is output. On the other hand, 
if E > 0.5L, the result is deemed unreliable. 

D. Statement of Main Result 

The analysis in this section leads to the following result. 

Theorem 1: Consider the transmission of a codeword from 
an LDPC code drawn at random from the ensemble C° over 
an MBIOS channel. Fix the proximity gap (5 > and the 
expurgation depth 7 > 0. Then the probability of frame 
eiTor with BP decoding given that the AMLC dHJ-® holds is 
upper-bounded by (l25ll-(l26]l. This bound holds with confidence 
level ^(X, e) which can be obtained using the L Monte Carlo 
experiments, as detailed in Algorithm [T] 

IV. Numerical Results and Discussion 

Figure 1 shows a comparison between the frame error rate 
(FER) obtained by a simulation of the BP decoder over the 
binary symmetric channel (BSC) and the DS2 bound ( |26] l. 
calculated for various values of 5. In this example, we consider 
the ensemble C" of (3,4)-regular LDPC codes with block 
length TV — 1000. For the calculation of the DS2 bound and 
the distance spectrum we use 7 = 20 as the expurgation depth. 

We conducted two experiments to determine the confidence 
level of the bound, using Algorithm [T] In the first experiment, 
150 randomly-generated codes were tested over a BSC with 
crossover probability p = 0.14. In the second experiment, 
600 randomly-generated codes were tested over a BSC with 
p = 0.1. In both experiments, all the codes belonged to 
the ensemble C. The results of the first experiment are 
summarized in Table U These results indicate that in this 
case, the null hypothesis dSOl ) can be rejected with very high 
confidence level even for S ~ 0. Consequently, the conditional 
frame error probability given that the the AMLC holds for 
(5 = 0, is (with very high confidence) lower than 3 • 10^^, 




Fig. 1. A comparison between simulated frame eiTor rate of the BP decoder 
and tlie DS2 bound (26), assuming the AMLC, for the (3,4)-reguIar LDPC 
ensemble and various values of <5. 
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0.1667 


7.13 ■ lO-'^^ 


5 


0.0733 


4.95 ■ lO-^'' 


10 


0.02 


1.6- 10-42 


20 





7- 10-46 



TABLE I 

Confidence level bounds for different values of 5 



which is about 1000 times lower than the simulated frame 
error rate at p = 0.14. In the second experiment, both the BP 
and LP decoders succeeded in decoding all transmissions, and 
thus in Algorithm [T] we get e 0. This puts the confidence 
level of all the DS2 bound curves in Figure 1 at an extremely 
high level of 

^(600,0) = 1 - 2"^°° (31) 

In this case, the conditional frame error probability given 
that the AMLC holds is more than 7 orders of magnitude 
smaller than the simulated frame error rate (the difference 
between the BP curve and the 5 — curve for p ~ 0.1). The 
confidence level in this case is also much higher than in the 
first experiment. Due to the high confidence levels observed in 
the first experiment with p = 0.14, the confidence level result 
of the second experiment with p — 0.1 is not surprising, and 
in general we expect the confidence level to increase as the 
channel noise level decreases. 

The strength of this result is that it demonstrates that the LP 
decoder can provide the BP decoder with extra error detection 
capability. This capability is especially useful in applications 
where a codeword should be rejected unless it is decoded 
correctly, and rejection should occur with high probability 
(as in data applications requiring high reliability). Achieving 
codeword reliability results of this order via simple Monte 
Carlo simulation would require very long simulation runs. In 
fact, our technique for upper bounding the frame error rate 
given that the AMLC property holds, has a common feature 
with the importance sampling method, since both attempt 
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to alleviate the computational burden associated with simple 
Monte Carlo simulation. We also note that in both experiments 
described above, the AMLC was satisfied in a large percentage 
of the trials, implying that it is not only capable of increasing 
the reliability of the decoder output, but it also does so very 
frequently. Using the AMLC provides an alternative to external 
error-detection codes, such as cyclic redundancy checks, which 
cause some coding rate loss. This comes at the expense of extra 
processing, in the shape of an LP decoder at the receiver We 
note that this decoder can be implemented in linear time [jS). 
There is also a one-time task of computing the confidence 
level, which can be performed off-line. The computational 
complexity of calculating the lower bound [9 | on the minimum 
distance LB{C) is quadratic in the block length. Thus the task 
of obtaining a confidence level using Algorithm[T]is performed 
with complexity 0{N'^L), where L is the number of simulated 
blocks. We also note the following points. 



standard iterative decoding schemes 0. The same goes for 
nonbinary LDPC codes when represented using the LP 
formulation proposed by Flanagan et al. |17|. 

Finally, it may be observed that our bound can be improved 
by any method which tightens the LP relaxation, e.g., the 
check node merging technique lifting methods |1|, and 
others. By using any of these methods, we can obtain a 
vector A such that P(A) is larger than that obtained by the 
standard LP decoder, essentially because the optimization ([T]) 
is performed over a smaller domain. The result is that for any 
BP-decoded codeword c, we can use a smaller value of & in 
the AMLC (|9]l, which gives an exponential improvement in 
the DS2 bound, as can be clearly seen in (I25]i-(l26ll. 



V. Conclusion 



It is possible to tune the AMLC result to obtain different 
error rates and confidence levels by varying the value of 
the proximity gap 5 and the expurgation depth 7. Higher 
values of 5 will produce higher values of the DS2 bound 
(this is evident from Figure 1), but on the other hand will 
increase the confidence level (as can be seen in Table 
as the requirement d?) becomes more lax. Higher values 
of 7 will yield lower values of the DS2 bound. This, 
however, comes at the expense of a lower confidence level 
because while running Algorithm [T] more codes will be 
rejected as having low minimum distance. 
One may observe that in the example above, the AMLC 
result is applied to a random selection of a code from an 
ensemble. In many applications, it would be desirable to 
apply the AMLC result to a specific code rather than an 
ensemble. The difficulty is that while ensemble averages 
of distance spectra are typically known or can be easily 
upper-bounded, for specific codes this is not the case 
in general. Naturally, if one obtains for a specific code 
the exact distance spectrum (or an upper bound), it is 
straightforward to plug it in the DS2 bound (l25i-(l2&i. 
Another alternative is to use known concentration results 
ifTSll for the distance spectrum which enable one to give 
upper bounds on the distance spectrum of a specific code, 
which themselves hold with some confidence level. This 
confidence level can be integrated with our confidence 
bound ^(L, e). The result would be a looser bound (as 
compared to (l25Tl-(l26Tl) which applies with confidence 
level worse than ^(L, e), but it would apply to specific 
codes. 

Application of the AMLC result is not restricted to the 
BP decoder The result extends trivially to any decoder 
which satisfies the symmetry condition (|32] |. In particular, 
this condition is fulfilled by standard message-passing 
algorithms, e.g., min-sum, Gallager-A, Gallager-B. 
In a more general context, the AMLC result can be 
applied to any LP formulation. In particular, the LP 
program proposed by Feldman [16] for general Turbo 
codes can be used to achieve better error detection under 



A new property, the approximate maximum-likelihood cer- 
tificate, is introduced. This property of a BP-decoded code- 
word enables to increase its reliability, i.e., to increase the 
error detection capability. This is achieved for LDPC codes 
using tools related to linear programming decoding, including 
a recently-proposed algorithm for finding a lower bound on the 
minimum distance of a specific code which serves to improve 
the result. By applying the AMLC in the error floor region, it 
was demonstrated that the property can imply a frame error 
rate several orders of magnitude lower than a simulated error 
rate. While the increased frame error detection capability only 
holds with a certain confidence level, it was shown that this 
level is extremely high in the error floor region. 



Appendix A 
Proof of Lemma[T] 

Consider first the BP decoder. From the symmetry of the 
BP algorithm over MBIOS channels 1 181 , we know that 



e,((-ir"-y) = 



c«(y), 
1 - Ci(y), 



= 
= 1 



(32) 



where (— l)''™ is a vector of ±1 corresponding to the code- 
word Cm, the multiplication (— l)*^™ -y is componentwise, and 
Ci (resp. Cm,i) is the z'th bit of c (resp. c,„). Now consider the 
LP decoder, which is used to produce the vector A. Fix e > 0. 
The vector A = satisfies the symmetry condition (see 

Lemma 6]) 



A.((-ir 



yj = 



A.(y), 

1-A,(y), 



= 

= 1 



(33) 



For standard parallel concatenated Turbo codes, no expurgation is needed 
because all codes in the ensemble do not have codewords of very low weight. 
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Now, 



where (c)^ is the i'th bit of codeword c. We now have 



Pr (c e AMLC((5) | c„i trans.) 

= Pr (P (c) - P (A) < (5, c e C I c„ trans.) 



Pr I ^ lo, 



(c.(y) 



Pr I ^ lo, 



Ai(y)) < (5, c e C I c„ trans. 

/Q(y.|o) 



(-1)<^- (£,((-!)'=". y) 



VQ(y.|i). 

Aj((-l)''"' • y)) < 5, c e C I trans. 

Q{y^W 



c G C I trans 



= Pr (P(c) - P(A) < 5, c £ C I trans.) 
= Pr (c e AMLC((5) | ti-ans.) 



where 

• the first inequahty is by the definition (|9|l. 

• in the second equaUty we use the definition (|5]l and stress 
the dependence on y. 

• in the third equahty we use the symmetry of the channel 
as well as the symmetry of the BP decoder (l32t . 

• in the fourth equality we use the symmetry of the LP and 
BP decoders (dS^l,®). 

• in the fifth equality we again use the definition (|5]l. 

• the final equality is again by the definition (|9]l. 

The above series of equalities hold for all m. This proves the 
claim. 



Appendix B 
Proof of lemma[2] 



For any two codewords ci and C2, define the sets 



-4i(ci,C2) = {i : (ci)j = , (C2)j = 1} 
A2(ci, C2) = {^ : (ci), = 1 , (C2), = 0} (34) 



Pr( ^^/^ :7g^e^>l 
C 7^ C™ Q(y I c,„) 



Cm trans. 



Q{V^ I 0) 




(35) 



Pr 



3c' G C Q(y I C) , 

c't^O ■ Q(y|0)' 



c = c, 

trans. 



where 



« the first equality is due to the definition (l34l i of the sets 
Ai and A2. 

• the second equality is due to the same definitions of Ai 
and A2 as well as the symmetry of the channel (Q{y\x) — 

Q{~v\i-x)). 

• the third equality is due to the symmetry of the channel. 

• the final equality, which is the desired result, is due to 
the linearity of the code. 

Appendix C 
Optimization of the DS2 bound 

Consider the DS2 bound (|25ll for fixed ft.. Let First, 
rewrite the bound in exponential form as 



Pi{h) < e 



Eos2{S, /3, p, A, ^(0) ^ ^-pX-^ln {A 



-p(l-/3)ln l^ij{y)^--^Q{y\0)^j 

-p/31n |^^^(y)^-^Q(2/|0)^Q(y|l)^j 

Assuming fixed values of f3 and S, the exponent 
Eosii^, P, P, A, should be maximized over 

A>0, 0<p<l, |7/^(y) :^V(y) = l| (36) 

For fixed values of A and p, we use calculus of variations to 
find the optimum tilting measure ip; this analysis yields the 
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optimality condition 



T,ytpiy)^ ^51 (y) 



^ m-j;)92{y) ' 
V'(y)^~^52(y), 

where /i is a Lagrange multiplier and 
The solution to ( [37] l is given in the following implicit form 

^(y) = C(5i(y) + «52(y))'' = CO(2/|0) 



[12] R. G. Gallager, Information Theory and Reliable Communication, Wiley, 
New York, 1968. 

[13] R. G. Gallager, "Low density parity checlc codes," IEEE Transactions 

on Information Theory, vol. 8, pp. 21-28, Jan. 1962. 
[14] D. Burshtein and G. Miller, "Asymptotic enumeration methods for 
analyzing LDPC codes," IEEE Transactions on Information Theory, 
vol. 50, no. 6, pp. 1115-1131, June 2004. 
/i = (37) [15] V. Rathi, "On the Asymptotic Weight and Stopping Set Distribution of 
Regular LDPC Ensembles," IEEE Transactions on Information Theory, 
vol. 52, no. 9, pp. 4212^218, September 2006. 
[16] J. Feldman, "Decoding Error-Correcting Codes via Linear Program- 
ming," September 2003, PH.D. Thesis, MIT. 
[17] M. F. Flanagan, V. Skachek, E. Byrne, and M. Greferath, "Linear- 
programming decoding of nonbinary linear codes," IEEE Transactions 
on Information Theory, vol. 55, no. 9, pp. 4131^154, September 2009. 
[18] T. Richardson and R. Urbanke, "The capacity of low-density parity- 
check codes under message-passing decoding," IEEE Transactions on 
Information Theory, pp. 599-618, February 2001. 



^"Wy\^)) 



where 



1-/3 



The appropriate normaUzing constant C is given by 



c 



(38) 



(39) 



To find the optimized tilting measure, we solve (1381 1 numeri- 
cally, and determine C, by ( [39] l. The optimal values of A and 
p are then found numerically. 
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